An on-the-fly approach optimized switching: A special case of 
free energy simulation under nonequilibrium feedback control 
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We discuss the optimized switching free energy simulations in an analogy with the systems which 
are driven under nonequilibrium feedback control. We find an on-the-fly simulation approach of 
switching optimization is a special case of the nonequilibrium process under feedback control. In 
this approach, the switching rate is allowed to vary during the simulation and the optimization is 
done on-the-fly by utilizing the part of the simulation outcomes as the feedback information. In 
such a case, one should use generalized Jarzynski equality under nonequlilibrium feedback control 
for free energy calculation instead original Jarzynski equality. 
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There are very few relations in statistical dynamics has 
been used to calculate the equilibrium thermodynamics 
properties for the systems which are driven arbitrarily 
far-from-equlibrium [l|-|3| . One of these is the Jarzynski 
equality |l| which relates nonequilibrium measurements 
of the work done on the system to equilibrium free en- 
ergy differences. The second law of thermodynamics can 
be quantitatively described by the fluctuation theorem 
which are closely related to the Jarzynski equality [l|, l2] ■ 
A system initially at equilibrium with temperature (in- 
verse) /3 = l/ksT is externally driven from its initial 
state to final state by nonequilibrum process satisfies the 
detailed fiuctuation theorem, 
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and its integrated version, the Jarzynski equality 
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Where W denotes the work performed on the system, 
AF is the free energy difference of the system between 
its final and initial equilibrium states and P{±W) is the 
work probability distribution in forward {+) and reverse 
(-) direction. This relationship is widely used in experi- 
ments y, |5| as well as simulations [6| in many branches 
of Science (see, eg.|7J). 

Various experiments and simulations has been per- 
formed by adopting a suitable time-dependent driving 
scheme described by an external control switching pro- 
tocol. Even though the Jarzynski equality is valid for 
any time-dependent driving scheme, the efficiency of a 
nonequilibrium switching simulation which use Jarzyn- 
ski equality to estimate precise free energy difference 
is depends on the switching function. A well selected 



switching function can significantly minimize the asso- 
ciated dissipated work Wd = W — AF and reduce the 
computational cost of nonequilibrium free energy simu- 
lation [i-[i3|. 

In simulations, one has to search for the switching pro- 
tocol that minimizes the mean work required to drive 
the system from one given equilibrium state to another 
in finite time such that the exponential average should 
provides precise free energy estimates |8l4ld|. Since the 
optimized switching protocol minimizes the average work 
[lO|, most nonequilibrium free energy simulations which 
use Jarzynski equality employ optimized switching func- 
tion 11| . One simple and reasonable method of optimiza- 



tion utilizes the informations obtained from an ensemble 
of short simulation which are carried out at relatively 
low switching speeds, usually within the linear response 
regime [ll| . Once the optimized control switching pa- 
rameter has been estimated, the long run free energy 
simulations then performed with the optimized switch- 
ing function at a slower speed. 

In a recent paper [12| . an on-the-fly approach is pro- 
posed for estimating an efficient switching function dur- 
ing a single nonequilibrium free energy simulation. In 
this approach, the part of the simulation carried out by 
allowing the switching function to vary and the remain- 
ing part continued with the optimized switching function 
which has been estimated on-the-fly. In this approach, 
the informations obtained from the earlier part of the 
simulation outcomes are used to estimate the optimized 
switching function. Here, the informations used for opti- 
mization are the switching rate, the variance of the simu- 



lation outcomes and its autocorrelation function ll|, [12 1 . 
It has been shown very recently that the accuracy of the 
free energy estimates also depends on shape of the switch- 
ing protocol |13{ . 

In most of the optimization problems 
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(feedback) informations about the physical system plays 
crucial role in a direct or indirect way to calculate the op- 
timized switching functions. This can provide the natural 



link between the switching optimization and the informa- 
tions. In this paper, we discuss the optimized switching 
free energy simulations in an analogy with the systems 
which are driven under nonequilibrium feedback control 
14| and argue that the work value obtained from an 
on-the-fly approach in general does not satisfy Jarzyn- 
ski equality. 

The evolution of the physical systems can be modified 
or controlled by repeated operation of an external agent 
called controller [IJ, |l5| . In contrast to open loop con- 
troller which operates on the system blindly, the feedback 
or closed loop controllers use information about the state 
of the system. The feedback is the process performed by 
the controller of measuring the system, deciding on the 
action given the measurement output, and acting on the 
system 16|. For example, in a single molecule Atomic 
Force Microscopy experiment, the external agent is an 
electric feedback circuit which detects the motion of the 
cantilever and manipulate the control force proportional 
to its velocity [171] . The proper utilization of the informa- 
tion about the state of the system in feedback control ef- 
fectively improves the system performance [14l4l8{ . How- 
ever, the presence of feedback control in physical system 
modifies both the Jarzynski equality and the fluctuation 
theorem |17| . 

Recently, the Jarzynski equality is generalized to an 
experimental condition in which the system is driven be- 
tween two equilibrium state via nonequilibrium process 
under forward feedback control [19| . At a given time, the 
controller measure the partial state of the system. The 
result of the measurement determines the action the con- 
trol will take. The additional information on the system 
provided by the measurement further determines the sys- 
tem states. The equilibrium free energy difference for the 
driven system (which locally satisfies detailed fluctuation 
theorem) under nonequilibrium feedback control can be 
calculated from the generalized Jarzynski equality [19J 
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1, 
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where a — P{W — AF) and / is the mutual information 
measure obtained from the feedback controller |19|. The 
average is taken from the distribution of work in forward 
direction with feedback control. The work distribution 
with feedback control is computed without the knowledge 
of internal details of the controller [19J . 

Let PiW,I) = P{W\I)P{I) be the (joint) probabil- 
ity distribution of work with feedback information. The 
above equality, Eq.Q, can be written as. 
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dW dl = 1, 
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where P{W\I) is the conditional probability for obtaining 
the outcome W given the mutual information measure /. 
Since the controller often measure the partial state of 
the s;retem, the changes in feedback information measure 
16|,|20| occurs with probability P{I). 



The feedback control enhances our controllability of 
small thermodynamics systems 1J-[18|. In free energy 
simulations this feedback mechanism can be very help- 
ful for sampling rare trajectories for precise free energy 
difference calculations. Since the work that performed 
on a thermodynamic system can be lowered by feedback 
control 18|, |l9[ , we can say that using proper feedback 
mechanism in a free energy simulation is equivalent to 
carry out an optimized switching free energy simulation 
which minimizes the average work. In this perspective, 
we notice that an on-the-fly approach is a special case of 
nonequilibrium process under feedback control as follows. 

In simulations, the free energy difference between the 
two equilibrium states can be calculated in general by 
pulling the sytem from one equilibrium state to another 
state along a switching path. The path connecting the 
two states will be parameterized using the variable A, 
with < A < 1. The switching rate describes the nature 
of the switching process to be an equilibrium (inflnitely 
slow) or nonequilibrium (fast). In an on-the-fly approach, 
up to the history length time, th [l^, the switching func- 
tion parameter. At, vary during the simulation. The 
simulation outcomes obtained within th are used as a 
feedback information to estimate the optimized switch- 
ing function, X^. The remaining part of the simulation 
carried out with this fixed AJ". This is equivalent to say, 
at time th, the (un- known) controller 23| measure the 
partial state of the system and determines the optimized 
switching function AJ". Once the switching function to be 
optimized, the outcome of the remaining part (after th) 
of the simulation is performed by fixed feedback control 
with feedback information measure, /(AJ'). 

In this aspect, the feedback information in an on-the- 
fly approach can be characterized by the mutual infor- 
mation measure [l9|, /i = for t < th and I2 — -f(AJ') 
for t > th- From a single measurement of the controller, 
at th the mutual information measure I{X^) is obtained 
over all simulation outcomes for t < th- The term mutual 
information is appropriate in the sense that in this ap- 
proach, the optimized switching function A^ depends on 
the feedback information measure I{X^) which in turn 
depends on Aj for further simulation. In an on-the-fly 
approach, the switching function is fixed at AJ" after th 
which indicates that the controller measure the partial 
state of the system only once (at th) for system control 
(switching optimization) instead of many times in gen- 
eral. In this sense, we can say that an on-the-fly approach 
is a special case of nonequilibrium process under feedback 
control. 

It should be noted that, in systems with feedback con- 
trol the simulation/experimental outcomes are different 
from controller measurement outcomes. A single out- 
come of the controller measurement is a result of ana- 
lyzing many realizations of the simulation/experimental 
outcomes. That is the controller utilizes the part of 
the simulation/experimental outputs and calculate its 



measurement outcome, see ref.[15|. In an on-the-fly ap- 
proach, the controller utilizes all simulation outcomes for 
t < tfi and obtained the optimized switching function X^. 
Due to the single measurement of the controller at t^, 
work outcomes of the simulation associated with mutual 
information measure /i = for t < th and I2 = I{X.f) 
for t > th- Since the equality, Eq.©, does not depends 
on the internal details of the controller 16|, ll9| and the 
controller made measurement only once at i = i?i, Eq.(j4]) 
can be written in this case as 



macromolecular systems which has many folding inter- 
mediates 2l|. Finally, If one can carry on-the-fly ap- 
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where, 
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P{W\h) e-" dW, (i = 1,2). 
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In contrast to the original Jarzynski equality, {e^") = 
1, if we measure (e""^) in the presence of the feedback 
control, the calculated value is differ from unity [19| 
which is given as 



{e'") 



7, 



(7) 



where 7 is a measure which characterizes the efficacy of 
the feedback control [19J. Hence, in an on-the-fly ap- 
proach, the free energy difference obtained from original 
Jarzynski equality [l2| is in general inappropriate. In 
this approach one should use the generalized Jarzynski 
equality with proper feedback information measure /(AJ") 
provided the simulation process should satisfy the local 
detailed balance or the detailed fluctuation theorem [19| . 
Since the switching function under feedback control de- 
pends on the controller measurement outcomes |19j , if we 
perform nonequilibrium simulation under feedback con- 
trol, the different trajectories will have different switching 
function. These kind of different switching function has 
been incorporated in an on-the-fly approach simulation 
|12| clearly confirm that this approach is a special case 
of free energy simulation with feedback control. Thus, 
the work value obtained from an on-the-fly approach in 
general does not satisfy original Jarzynski equality. The 
un-noticed sign of violation of original Jarzynski equal- 
ity for those simple systems studied in an on-the-fly ap- 
proach |12l | is in question. In fact, the free-energy es- 
timate by the original Jarzynski equality for those sys- 
tems studied in an on-the-fiy approach seems to be pretty 
good which can makes a general impression that the gen- 
eralized Jarzynski equality does not needed. The main 
motivation for our analogy argument of on-the-fly ap- 
proach with nonequilibrium process under feedback con- 
trol is that the stated conclusion of the un-noticed sign 
of violation of original Jarzynski identity in an on-the-fly 
approach is not definite for all systems (see, ref. 17J|'). 



particularly, the folding and unfolding free energy simu- 
lations studies for complex macromolecular systems. It 
would be interesting to carry such an investigation on 



proach simulation studies on complex systems and in- 
sist to use original jarzynski equality for free enregy es- 
timates, within the confident level of certain hypothe- 
sis [22| one should properly test the violation of original 
Jarzynski equality. 

In conclusion, our analogy argimient of the switching 
optimization with nonequilibrium process under feedback 
control indicates that an on-thc-fly approach switching 
simulation is a special case of a nonequilibrium free en- 
ergy simulation with feedback control. In such a case, 
using generalized Jarzynki equality under nonequilibrium 
feedback control is appropriate for free energy difference 
calculation instead original Jarzynki equality. 
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